“Big data” is a broad term for datasets so large or complex that traditional data processing applications are often inadequate. For example, a social networking system can run several application services that pertain to big data. The term “big data” also often refers to the use of predictive analytics or other methods to extract values from data. For example, analysis of datasets can find new correlations, trends, patterns, categories, etc. Such analysis rely on machine learning.
A typical machine learning workflow may include building a model from a sample dataset (referred to as a “training set”), evaluating the model against one or more additional sample datasets (referred to as a “validation set” and/or a “test set”) to decide whether to keep the model and to benchmark how good the model is, and using the model in “production” to make predictions or decisions against live input data captured by an application service. The training set, the validation set, and/or the test set can respectively include pairing of input datasets and expected output datasets corresponding to those input datasets.
A variety of web-based or mobile applications often rely on machine learning models to process large and complex “big data” to provide application services (e.g., personalized or targeted application services) to a large number of users. There is frequently a need for higher accuracy and/or consistency models while the requirements of these models are ever evolving. The training and evaluation of these models nevertheless take time and are typically the manual burdens of one or more developers or analysts. Developers and/or analysts have to manually update the parameters and training sets for the models in order to keep the models current, but are often challenged by having to manually analyze the models to determine betters ways of evolving the models.
The figures depict various embodiments of this disclosure for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of embodiments described herein.